Repository navigation
feat(cli): record which config key failed in config-load telemetry (CLI-2912) - #7468
Merged
Coly010 merged 4 commits intoNov 3, 2026
Conversation
…LI-2912) Failed cli_command_executed events classified invalid_config now carry config_key, the config key path behind the failure. Keys come from a closed vocabulary built from the config schema and registry, so values, file paths and user-chosen names never reach telemetry.
…erry/cli-2912-record-which-config-key-failed-in-config-load-telemetry # Conflicts: # apps/cli/src/shared/telemetry/event-catalog.ts # apps/cli/src/telemetry/command-telemetry-attributes.ts
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🤖 AI Review
Verified all three findings against the checked-out code and trusted conventions. After merging the overlapping stale-key reports, two minor telemetry attribution issues remain confirmed.
Findings
| Severity | Location | Category | Sources | Claim |
|---|---|---|---|---|
| 🟡 MINOR | apps/cli/src/config/cli-config-values.layer.ts:71 |
telemetry-accuracy |
claude+codex | A recovered config failure can leave its config_key attached to a later, unrelated invalid_config failure. Keyless failures do not clear the previously recorded key, and the final telemetry gate checks only the error category. |
| 🟡 MINOR | apps/cli/src/config/cli-config-failure-key.ts:62 |
telemetry-accuracy |
codex | An unselected remote named after a root config section can cause config_key to identify the remote name instead of the failing section. For example, a remote named auth with db = "invalid" reports auth instead of db. |
Stats
Claude findings: 1 · Codex findings: 2 · Confirmed: 2 · Refuted: 0 · Uncertain: 0
Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: auto · Workflow run
This review runs once per PR. A maintainer can request another with a /ai-review comment.
Contributor
CLI benchmark (Lightweight Commands): no changes foundCommit
API calls: unchanged for all 7 commands. Binary size: 134.1 MB, up 12.0 KB from 134.0 MB. Details: timing ranges, API calls per command, slowest phasesTiming ranges
API calls per command
Slowest phasesNo command moved beyond noise, so there are no phase deltas. Comment |
…ovenance A keyless config failure or a successful load or lookup now clears the recorded key, so a recovered failure cannot label a later one. Remote-block schema failures are detected from the decode itself instead of by comparing prefix depth, so a remote named after a config section reports the failing section.
A successful load or lookup with no earlier failure no longer records an empty config_key, so commands that assert their recorded attributes see only their own.
mdonnalley
approved these changes
Nov 3, 2026
Coly010
deleted the
columferry/cli-2912-record-which-config-key-failed-in-config-load-telemetry
branch
November 3, 2026 15:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A failed
cli_command_executedevent shows that a command failed on config (error_category: invalid_config,error_fingerprint: tag:StartConfigLoadError), but not which key caused it. So we can't tell how many config-load failures come from the removed[inbucket]section and how many from everything else.This adds a
config_keyproperty to those events.CliConfigValues.loadandResolvedCliConfig.getrecord the key through the existingrecordCommandTelemetrysink when they fail. It has to be recorded at load time because command wrappers such asStartConfigLoadErrorkeep only the message and drop the original errorCliConfigRemovedKeyError: every removed key, sorted and comma-joinedCliConfigValueError: its pathCliConfigParseError: the first issue path, when the cause is a schema errorfunctions.my-fn.verify_jwtrecordsfunctionsand a customauth.external.<name>recordsauth.externalwithCommandTelemetryonly addsconfig_keywhen the event'serror_categoryisinvalid_config, so a config failure that a command swallowed can't label a success or an unrelated failureapps/cli/docs/analytics.mddocuments the new propertyThe list of removed keys is kept in the CLI rather than exported from
@supabase/config, so this change stays out of that package's release train.Linked issue
Closes CLI-2912
🤖 Generated with Claude Code